A Proposed Architecture for Continuous Web Monitoring Through Online Crawling of Blogs

摘要

Getting informed of what is registered in the Web space on time, can greatlyhelp the psychologists, marketers and political analysts to familiarize,analyse, make decision and act correctly based on the society`s differentneeds. The great volume of information in the Web space hinders us tocontinuously online investigate the whole space of the Web. Focusing on theconsidered blogs limits our working domain and makes the online crawling in theWeb space possible. In this article, an architecture is offered whichcontinuously online crawls the related blogs, using focused crawler, andinvestigates and analyses the obtained data. The online fetching is done basedon the latest announcements of the ping server machines. A weighted graph isformed based on targeting the important key phrases, so that a focused crawlercan do the fetching of the complete texts of the related Web pages, based onthe weighted graph.